System and method for fusing three dimensional image data from a plurality of different imaging systems for use in diagnostic imaging

ABSTRACT

A multi-modality cancer screening and diagnosis system is provided that allows cancer screening and diagnosis of a patient using at least two different and sequential three-dimensional imaging techniques without patient repositioning. The system includes a first three-dimensional image acquisition device, a second three-dimensional image acquisition device having a probe with a transmitter mounted thereon, and a positioning paddle for positioning and immobilizing an object to be imaged during the cancer screening and diagnosis procedure. The positioning paddle is designed to facilitate visualization of the breast in both three-dimensional modalities without movement of the patient, and preferably is designed to position the patient with comfort during a diagnosis procedure which uses both imaging modalities.

RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/904,689, now U.S. Pat. No. 10,729,403, filed Feb. 26, 2018, entitled“System and Method for Fusing Three Dimensional Image Data from aPlurality of Different Imaging Systems for Use in Diagnostic Imaging”;which is a continuation of U.S. application Ser. No. 13/325,495, nowU.S. Pat. No. 9,901,320, filed Dec. 14, 2011, entitled “System andMethod for Fusing Three Dimensional Image Data from a Plurality ofDifferent Imaging Systems for Use in Diagnostic Imaging”; which isrelated to, and claims priority under 35 U.S.C. 1.119 (e) to U.S.Provisional Application Ser. No. 61/422,991, filed Dec. 14, 2010,entitled “A Tomo/Ultrasound Fusion Device for Diagnostic Imaging,” thedisclosures of which are hereby incorporated by reference herein intheir entireties.

This application is related to U.S. patent application Ser. No.12/954,633, filed on Nov. 25, 2010 by the same assignee as the presentapplication, and entitled “SYSTEMS AND METHOD FOR TRACKING POSITIONSBETWEEN IMAGING MODALTIES AND TRANSFORMING A DISPLAYED THREE-DIMENSIONALIMAGE CORRESPONDING TO A POSITION AND ORIENTATION OF A PROBE,” whichclaims priority to U.S. Provisional Application 61/264,743, filed Nov.27, 2009 and U.S. Provisional Application 61/394,734, filed Oct. 19,2010, all of which are incorporated herein by reference.

BACKGROUND

Medical imaging devices provide non-invasive methods to visualize theinternal structure of a patient. Such non-invasive visualization methodscan be helpful in treating patients for various ailments. For example,the early detection of cancer in a patient can be important in treatingthat patient. For most cancers, when detected at an early stage, thesurvival probability of the patient can increase.

In the U.S. breast cancer mortality is second only to that of lungcancer. Because of its role in early tumor detection, mammography hasbecome the most commonly used tool for breast cancer screening,diagnosis and evaluation in the United States. A mammogram is an x-rayimage of inner breast tissue that is used to visualize normal andabnormal structures within the breasts. Mammograms provide early cancerdetection because they can often show a breast lumps and/orcalcifications before they are manually palpable. One drawback ofmammography is that it provides only a two-dimensional representation ofa compressed breast, and as a result masses which are hidden byintervening structures may not always be readily discernible on amammogram.

Tomosynthesis systems, which are x-ray systems for obtaining a threedimensional image volume of a breast, have recently been developed foruse in breast cancer screening. One such tomosynthesis system, theSelenia® Dimensions® breast tomosynthesis system, is provided byHologic, Inc., of Bedford Mass., the assignee of the present invention.In contrast to typical mammography systems, the tomosynthesis systemacquires a series of x-ray projection images, each projection imageobtained at a different angular displacement as the x-ray sourcetraverses along a path over the breast. Reconstructed tomosynthesisslices reduce or eliminate the problems caused by tissue overlap andstructure noise in single slice two-dimensional mammography imaging.Digital breast tomosynthesis also offers the possibility of reducedbreast compression, improved diagnostic and screening accuracy, fewerrecalls, and 3D lesion localization. Examples of breast tomosynthesissystems are described in U.S. Pat. Nos. 7,245,694 and 7,123,684,commonly owned by the Assignee of this application and incorporated byreference herein.

While mammography (and now tomosynthesis systems) have become the ‘goldstandard’ for breast cancer screening, if the screening identifies alump or a mass the standard protocol recommends review of the patientusing a different imaging modality such as ultrasound imaging, tofurther characterize the mass or region of interest during breast cancerdiagnosis.

Ultrasound imaging, another non-invasive medical imaging technique, usessound waves, typically produced by piezoelectric transducers to image atissue in a patient. The ultrasound probe focuses the sound waves,typically producing an arc-shaped sound wave which travels into the bodyand is partially reflected from the layers between different tissues inthe patient. The reflected sound wave is detected by the transducer andconverted into electrical signals that can be processed by theultrasound scanner to form an ultrasound image of the tissue.

The typical procedure followed to obtain ultrasound images of apatient's breast involves positioning a patient in a supine positionupon a table, applying a gel or other acoustic couplant to the patient'sbreast, and passing an ultrasound transducer across the patient'sbreast. As the transducer traverses the breast, ultrasound images cantypically be viewed in real-time on a display of an ultrasound system.The ultrasound transducer may be either a hand-held transducer which ismanually manipulated by the imaging technician, or may be an automatedscanning device, such as that described in U.S. Pat. No. 7,731,662. Onedrawback of such methods lies in the fact that the breast is a verymalleable structure; the geometry and structures of the breast move andchange whenever the patient changes position. Thus, a mass which isreadily identified when a patient is positioned for imaging using afirst modality may be difficult to detect when the patient isrepositioned for examination using a second modality.

SUMMARY OF THE INVENTION

According to one aspect of the invention, a multi-modality cancerscreening and diagnosis system is provided that allows cancer screeningand diagnosis of a patient using at least two different and sequentialthree-dimensional imaging techniques without patient repositioning. Thesystem comprises a first three-dimensional image acquisition device, asecond three-dimensional image acquisition device having a probe with atransmitter mounted thereon, and a positioning paddle for positioningand immobilizing an object to be imaged during the cancer screening anddiagnosis procedure. Although the present invention is not to belimited, in one embodiment, the object to be imaged is a patient'sbreast, and the position of the patient for screening and diagnosis isan upright position. The positioning paddle is designed to facilitatevisualization of the breast in both three-dimensional modalities withoutmovement of the patient, and preferably is designed to position thepatient with comfort during a diagnosis procedure which uses bothimaging modalities.

According to another aspect of the invention, a method of examining anobject for diagnosis purposes includes the steps of positioning theobject prior to a diagnosis process using a positioning mechanism,acquiring a three dimensional image of the object using a first imagingmodality, displaying the three dimensional image of the object, and,during a diagnosis process, acquiring real-time three-dimensional imagesof the object using a transmitting probe and comparing the threedimensional images acquired using the first imaging modality to thethree dimensional images of the object acquired using the transmittingprobe. Following the diagnosis process the positioned object isreleased. In one embodiment the positioned object is a patient's breast,and a mechanism that is used to position the patients breast is apositioning paddle that enables image acquisition using multiplethree-dimensional imaging modalities while the patient remains in oneposition, and preferably while maintaining patient comfort.

Such an arrangements overcome the problems of breast cancer screeningand diagnosis in the prior art, which typically requires a patient tomove between the preferred, upright screening position to a supineposition for further diagnosis, often making it difficult for theradiologist to ensure that they have located the region of interestidentified during the screening process.

These and other aspects of the present invention will be described inmore detail with regard to the Figures identified below.

DESCRIPTION OF THE FIGURES

FIG. 1 is a block diagram of one embodiment of a system which permitsfusion of x-ray images and ultrasound images for breast cancerdiagnosis;

FIG. 2 is a side view an x-ray imaging gantry including a positioningpaddle according to the present invention, and having navigationhardware removably mounted thereon;

FIG. 3 illustrates in more detail an embodiment of a positioning paddlewhich may be provided on the gantry of FIG. 2 ;

FIGS. 4A and 4B illustrate an exemplary transmitter of a tracking systemwhich may be used in the present invention, the transmitter shownrespectively mounted on an ultrasound probe in FIG. 4A, and on a biopsyhand piece in FIG. 4B;

FIG. 5 illustrates the x-ray gantry of FIG. 2 , shown with an ultrasoundprobe having a tracking system transmitter mounted thereon;

FIG. 6 is an exemplary illustration of an ultrasound acquisition usingthe transmitting ultrasound probe shown in FIG. 5 ;

FIG. 7 is a flow diagram illustrating exemplary steps in a diagnosisprocess which integrates three dimensional x-ray data with real-timeultrasound data; and

FIG. 8 illustrates equipment which may be included in a radiology suiteto enable the performance of the method of FIG. 7 .

DETAILED DESCRIPTION

A multi-modality cancer screening and diagnosis system will now bedescribed which enables screening and diagnosis of a patient to beperformed using multiple three-dimensional image acquisition systemswhile maintaining constant patient positioning. A positioning paddle,suitable for use in two different and distinct imaging modalities, helpsto achieve this result. The system will be described with regard to thescreening and diagnosis of a patient's breast, using an uprightthree-dimensional x-ray system (an in particular, a tomosynthesisimaging system) as a first imaging modality, and a three-dimensionalultrasound imaging system as a second modality, although it should beunderstood that the present invention is not limited to these particularimaging modalities. For example, the present invention may be used withthree-dimensional images acquired via stereotactic x-ray acquisition,SPECT, PET, gamma, or other Computed Tomography image acquisitionsystems and the like.

It is understood that fusing images obtained from different imagingmodalities, and even with regard to breast cancer screening, is not newto the art. For example, in U.S. Pat. No. 5,474,072, entitled “Methodsand Apparatus for Performing Sonomammography” Schmulewitz describes amethod and apparatus for combining a mammography machine with anultrasound transducer, to generate ultrasound images that are inregistration with a mammogram. Drawbacks of the '072 patent include thefact that only two-dimensional data is obtained via the mammogram, andthat the path of the ultrasound probe is pre-determined, removing theopportunity for real-time manual manipulation.

U.S. Patent Application No. US20030194050, entitled “Multi modalityX-ray and nuclear medicine mammography imaging system and method”describes a multi modality imaging system that includes an X-ray imagingsubsystem and a nuclear medicine imaging subsystem, where the X-rayimaging subsystem may be a tomosynthesis subsystem. The system may beused for mammography imaging, such that the X-ray imaging subsystem andthe nuclear medicine imaging subsystem are adapted to image a breastcompressed by a breast compression paddle. Similar to the '072 patent,the system described in the '050 application does not allow forreal-time manual manipulation of the second imaging modality. Inaddition, both systems position the patient in compression, which maybecome uncomfortable for the patient during prolonged diagnosis.Further, the arrangement of the compression plates and imaging devicesin both systems preclude the ability to perform any interventionalprocedures, such as breast biopsy, when the patient is positioned withinthese configurations.

In contrast, as will become apparent from review of the detaileddescription, the present invention facilitates screening and diagnosisof the patient in comfort, using information from at least twothree-dimensional image acquisition devices, at least one of which maybe manipulated in real time. For example, referring now to FIG. 1 , anembodiment of system 100 is shown. System 100 includes an X-RAY imageacquisition device 10, a tracking system 15, an ultrasound imagingsystem 14, a navigation system 13 and a display system 16, allrepresentatively connected via communication network 12. It should benoted that, although the ‘systems’ are shown in FIG. 1 as functionalblocks, different systems may be integrated into a common device, andthe communication link may be coupled between fewer than all of thesystems; for example, the tracking system, navigation system and displaysystem may be included in an acquisition work station or a technologistwork station which may control the acquisition of the x-ray images in aradiology suite. Alternatively, the navigation and tracking systems maybe integrated into the ultrasound system, or provided as standalonemodules with separate communication links to the display, x-rayacquisition system and ultrasound system. Similarly, skilled personswill additionally appreciate that communication network 12 can be alocal area network, wide area network, wireless network, internet,intranet, or other similar communication network.

In one embodiment, X-Ray image acquisition system 10 is a tomosynthesisacquisition system which captures a set of projection images of apatient's breast as an x-ray tube scans across a path over the breast.The set of projection images is subsequently reconstructed to athree-dimensional volume which may be viewed as slices or slabs alongany plane. The three-dimensional volume may be stored locally on X-RAYimaging system 10 or in some embodiments in a Picture ArchivingCommunications System (PACS). Typically, the image format of the X-RAYimage is a DICOM format, however, skilled persons will understand thatother image formats can be used.

X-RAY imaging system 10 transmits the three-dimensional X-RAY imagevolume to navigation system 13 via communication network 12, where suchX-RAY image can be stored and viewed. Skilled persons will understandthat the X-RAY image of a patient can, in alternative embodiments, bestored locally on X-RAY imaging system 10 and accessed remotely bynavigation system 13 via communications network 12, and in otherembodiments can be stored on a server in communication with navigationsystem 13 via communications network 12. Navigation system 13 displaysthe X-RAY image obtained by X-RAY imaging system and once reconstructedfor display on navigation system 13 the X-RAY image can be reformattedand repositioned to view the image at any plane and any slice positionor orientation. In some embodiments navigation system 13 displaysmultiple frames or windows on the same screen showing alternativepositions or orientations of the X-RAY-image slice.

Skilled persons will understand that the X-RAY image volume obtained byX-RAY imaging system 10 can be transmitted to navigation system 13 atany point in time and is not necessarily transmitted immediately afterobtaining the X-RAY image volume, but instead can be transmitted on therequest of navigation system 13. In alternative embodiments, the X-RAYimage volume is transmitted to navigation system 13 by a transportablemedia device, such as a flash drive, CD-ROM, diskette, or other suchtransportable media device.

Ultrasound imaging system 14 obtains an ultrasound image of a tissue ofa patient, typically using an ultrasound probe, which is used to image aportion of a tissue of a patient within the field of view of theultrasound probe. Ultrasound imaging system 14 obtains and displays anultrasound image of a patient's anatomy within the field of view of theultrasound probe and typically displays the image in real-time as thepatient is being imaged. In some embodiments, the ultrasound image canadditionally be stored on a storage medium, such as a harddrive, CD-ROM,flash drive or diskette, for reconstruction or playback at a later time.

In some embodiments, navigation system 13 can access the ultrasoundimage, and in such embodiments ultrasound imaging system 14 is furtherconnected to communication network 12 and a copy of the ultrasound imageobtained by ultrasound imaging system 14 can be transmitted tonavigation system 13 via communication network 12. In other embodiments,navigation system 13 can remotely access and copy the ultrasound imagevia communication network 12, and in alternative embodiments, a copy ofthe ultrasound image can be stored on a server in communication withnavigation system 13 via communications network 12 and accessed remotelyby navigation system 13.

Tracking system 15 is in communication with navigation system 13 viacommunications network 12 and tracks the physical position in whichultrasound imaging system 14 is imaging the tissue of the patient. Insome embodiments, tracking system 15 can be connected directly tonavigation system 13 via a direct communication link or wirelesscommunication link. Tracking system 15 tracks the position oftransmitters connected to ultrasound imaging system 14 and providesnavigation system 13 with data representing their coordinates in atracker coordinate space. In some embodiments, tracking system may be anoptical tracking system comprising an optical camera and opticaltransmitters, however skilled persons will understand that any device orsystem capable of tracking the position of an object in space can beused. For example, skilled persons will understand that in someembodiments an RF tracking system can be used, comprising an RF receiverand RF transmitters.

Ultrasound imaging system 14 is configured for use with navigationsystem 13 by a calibration process using tracking system 15.Transmitters that are removably connected to the ultrasound probe ofultrasound imaging system 14 can transmit their position to trackingsystem 13 in the tracker coordinate space, which in turn provides thisinformation to navigation system 13. For example, transmitters may bepositioned on the probe of ultrasound imaging system 14 so that trackingsystem 15 can monitor the position and orientation of the ultrasoundprobe and provide this information to navigation system 13 in thetracker coordinate space. Navigation system 13 can use this trackedposition to determine the position and orientation of the transducer, anultrasound probe, relative to the tracked position of the transmitters.

In some embodiments, configuration occurs using a configuration tool,where its position and orientation can be additionally tracked bytracking system 15. During configuration the configuration tool contactsthe transducer face of the ultrasound probe of ultrasound imaging system14 and tracking system 15 transmits information representing theposition and orientation of the configuration tool in the trackercoordinate space to navigation system 13. Navigation system 13 candetermine a configuration matrix that can be used to determine theposition and orientation of the field of view of the ultrasound probe inthe tracker co-ordinate space, based on the tracked position of thetransmitters connected to the ultrasound probe. In alternativeembodiments, a database having configuration data of a plurality ofbrands or models of various ultrasound probes can be used to pre-load afield of view configuration into navigation system 13 duringconfiguration.

Once ultrasound imaging system 14 is configured with navigation system13, the tissue of a patient can be imaged with ultrasound imaging system14. During ultrasound imaging, tracking system 15 monitors the positionand orientation of the ultrasound probe of ultrasound imaging system 14and provides this information in the tracker co-ordinate space tonavigation system 13. Since ultrasound imaging system 14 has beenconfigured for use with navigation system 13, navigation system 13 isable to determine position and orientation of the field of view of theultrasound probe of ultrasound imaging system 14.

Navigation system 13 can be configured to co-register an ultrasoundimage with an X-RAY image. In some embodiments, navigation system 13 canbe configured to transform the position and orientation of the field ofview of the ultrasound probe from the tracker co-ordinate space to aposition and orientation in the X-RAY image, for example, to DICOMco-ordinates. This can be accomplished by tracking the position andorientation of the ultrasound probe and transmitting this positionalinformation in the tracker co-ordinate space to navigation system 13 andrelating this positional information to the X-RAY co-ordinate system.For example, in some embodiments, a user can select an anatomical planewithin the X-RAY image, and the user can then manipulate the positionand orientation of a tracked ultrasound probe to align the field of viewof the ultrasound probe with the selected anatomical plane. Oncealignment is achieved, the associated tracker co-ordinate spaceco-ordinates of the ultrasound image can be captured. Registration ofthe anatomic axes (superior-inferior (SI), left-right (LR) andanterior-posterior (AP)) between the X-RAY image and the trackerco-ordinate space can be determined from the relative rotationaldifferences between the tracked ultrasound field of view orientation andthe selected anatomical plane using techniques known to those of skillin the art.

This configuration further includes the selection of landmark within theX-RAY image, for example, using an interface permitting a user to selectan anatomical target. In some embodiments, the landmark can be aninternal tissue landmark, such as veins or arteries, and in otherembodiments, the landmark can be an external landmark, such as afiducial skin marker or external landmark, such as a nipple. The samelandmark selected in the X-RAY image can be located with the ultrasoundprobe, and upon location, a mechanism can be provided for capturingcoordinates of the representation of the target in the trackerco-ordinate space. The relative differences between the coordinates ofthe target in the X-RAY image and the co-ordinates of the target in thetracker co-ordinate space are used to determine the translationalparameters required to align the two co-ordinate spaces. The planeorientation information acquired previously can be combined with thetranslation parameters to provide a complete 4×4 transformation matrixcapable of co-registering the two co-ordinate spaces.

Navigation system 13 can then use the transformation matrix to reformatthe X-RAY image being displayed so that the slice of tissue beingdisplayed is in the same plane and in the same orientation as the fieldof view of the ultrasound probe of ultrasound imaging system 14. Matchedultrasound and X-RAY images may then be displayed side by side, ordirectly overlaid in a single image viewing frame. In some embodiments,navigation system 13 can display additional X-RAY images in separateframes or positions on a display screen. For example, the X-RAY imagecan be displayed with a graphical representation of the field of view ofultrasound imaging system 14 wherein the graphical representation of thefield of view is shown slicing through a 3D representation of the X-RAYimage. In other embodiments annotations can be additionally displayed,these annotations representing, for example, the position of instrumentsimaged by ultrasound imaging system 14, such as biopsy needles, guidancewires, imaging probes or other similar devices.

In other embodiments, the ultrasound image being displayed by ultrasoundimaging system 14 can be superimposed on the slice of the X-RAY imagebeing displayed by navigation system 13 so that a user can view both theX-RAY and ultrasound images simultaneously, overlaid on the samedisplay. In some embodiments, navigation system 13 can enhance certainaspects of the super imposed ultrasound or X-RAY images to increase thequality of the resulting combined image.

An exemplary Method and system which may be used to navigate between athree dimensional image data set and an ultrasound feed, and to aligncoordinate systems to enable display of common reference points isdescribed in further detail below, as well as in co-pending U.S. patentapplication Ser. No. 12/954,633, (hereinafter the '633 application)filed on Nov. 25, 2010 by the same assignee as the present application,and entitled “SYSTEMS AND METHOD FOR TRACKING POSITIONS BETWEEN IMAGINGMODALITIES AND TRANSFORMING A DISPLAYED THREE-DIMENSIONAL IMAGECORRESPONDING TO A POSITION AND ORIENTATION OF A PROBE”, which claimspriority to U.S. Provisional Application 61/264,743, filed Nov. 27, 2009and U.S. Provisional Application 61/394,734, filed Oct. 19, 2010, all ofwhich are incorporated herein by reference.

FIG. 2 illustrates an embodiment of an x-ray gantry which has beenmodified to incorporate components of the tracking system 15 and apositioning paddle 24 of the present invention. The gantry 200 includesan x-ray tube housing portion 22, an upright portion 23 and a detectorhousing portion 28. The upright portion is generally slideably mountedon a fixed base device (not shown), which allows the gantry to bepositioned below a patients breast to facilitate examination. The x-raytube housing portion is rotatably coupled to the upright portion 23,allowing x-ray images to be captured both in multiple orientations andusing multiple modalities (i.e., CC, MLO, mammography, tomosynthesis,sterotactic). Mounted within the x-ray tube housing is an x-ray source(not shown). An optical camera 20, which is part of the tracking system15, may be removably or fixedly mounted to the x-ray tube housing 22.One advantage of positioning the optical camera 20 in a fixed mannerrelative to the positioned breast is that it facilitates co-registrationof the three-dimensional volume and the ultrasound transducer. Animmobilization arm 26, is slideably mounted for movement along the Yaxis to an upright portion 23 of the gantry 200. The immobilization arm26 may include a latch 27 to permit coupling of interchangeable breastpositioning paddles. In FIG. 2 , a breast phantom 25 is shown positionedon the detector housing 28 using a breast positioning paddle 24 of thepresent invention. Prior to screening or diagnosis, a patient ispositioned, facing the gantry 200, and the patient's breast may beplaced on the surface of the detector housing. The compression arm 26moves downward, positioning the paddle 24 over the patient's breast. Thepressure used to position the patient's breast is sufficient only toimmobilize the breast to discourage movement of tissue during thescreening and diagnostic procedure.

As discussed above, during a tomosynthesis image acquisition procedure,the x-ray tube head is rotated along a path generally in the x plane,and x-ray projection images are captured at various points along thescan path of the x-ray tube. For example, referring to FIG. 2 , thex-ray tube head generally traverses in a direction normal to the reader,which has been indicated as the x-plane in FIG. 2 .

FIG. 3 illustrates the positioning paddle 24 in greater detail. As willbe described, the positioning paddle is suitable for use in at least twoseparate and distinct imaging modalities, i.e., image modalities thatimage using different sources, such as x-ray imaging and ultrasoundimaging. According to one aspect of the invention, a positioning paddlesuitable for three-dimensional image generation across multiplemodalities includes a material 30, positioned between a pair of opposingarms 31 a and 31 b, wherein each of the arms are attached to a base 33.The material is sufficiently sheer to be transparent to x-rays, andtherefore does not interfere with the x-ray image. In one embodiment,the material is formed from a porous fabric, for example a polyester ornylon blend fabric, such as tulle or the like. The material may beinelastic, or may be formed of a material with a limited amount ofelasticity. Characteristics of the material is that it should begenerally radiolucent, have sufficient porosity to enable acousticcouplant applied to a first surface to reach a second surface and be ofsufficient tensile strength and elasticity to immobilize a breast duringscreening and diagnosis. Additional desirable characteristics are thatthe material be generally non-abrasive so as to limit any discomfortexperienced by a patient undergoing treatment.

According to another aspect, the material is disposable. Various methodsof providing a disposable positioning paddle are within the scope of theinvention. Examples include arranging the material on a two sided framehaving mating edge, wherein the mating edges are slid into mating slotsof each of the arms 31 a and 31 b. Alternatively, a disposable orre-usable immobilization paddle may be provided which includes thematerial as part of the paddle. A switch, dial or other mechanism may beprovided on the arms or base of the paddle to increase the ‘tightness’of the material. The tightness may be adjusted after the material ispositioned over the patient's breast, before the material is placed onthe patient's breast, or a combination of both before and after. Forexample, in one embodiment a mesh screen may be slid into thereceptacles within the arm, a dial or other control may provide a firstadjustment to the tightness of the screen, the paddle may be movedvertically downward into contact with the breast, and subsequentadjustment to the screen may be used to achieve final immobilization.Alternatively, the mesh material may simply be brought into contact withthe breast to a desired compression, as typically done for mammographyscreening. In still a further embodiment, the arms 31 a, 31 b, may becoupled to the base 33 of the paddle to permit their movement along thepath generally represented by the arrows A in FIG. 3 . Theimmobilization arm of the gantry may be brought down towards thepatient's breast with the arms in position A′. When the immobilizationarm has reached the desired orientation relative to the patient'sbreast, the arms may then be brought down into the position shown inFIG. 3 , securing the material over the patient's breast and allowingfor any tightening that is desired to be performed.

In an alternate embodiment, the arms 31 a and 31 b are also adjustablealong the x-axis, as shown in the arrows B in FIG. 3 . Adjustment of thearms in this manner may allow for increased tension to be applied to thebreast in the chest area of the patient, while less tension is appliedto the nipple, as is sometimes desired for improved tissue capture.

As will be described in more detail below, one benefit of using amaterial of a porous nature is that it facilitates the application ofacoustic couplant during an adjunctive ultrasound procedure. However, itis envisioned that the mesh paddle shown in FIG. 3 may have advantagessimply for breast cancer screening, particularly for tomosynthesisscreening, which does not generally require the application of highcompressive forces such as is used during mammographic imaging. It isrecognized that in instances when it is not desired to follow atomosynthesis scan with an ultrasound review, it may be desirable toprovide a system wherein the mesh material may be simply rolled into oneof the arms 31 b, to provide sanitary material for the next patient.Thus, one embodiment of the invention envisions that the two arms 31 aand 31 b comprise roller arms, and mechanisms are provided on the armsor at the base for rolling the mesh material from one arm to the otherto provide sanitary material for each patient.

As mentioned above, the positioning paddle helps to enable fusiontomosynthesis images and ultrasound images because it allows the patientto remain in a single position during both imaging procedures. Becausethe patient remains in a fixed position, a one to one correspondencebetween the x-ray images and ultrasound image feed is made possible.This overcomes problems of the prior art, wherein a ultrasound diagnosiswas generally performed by changing a patients position, and the medicalprofessional could only guess that a particular region identified inultrasound corresponded to the region identified by x-ray.

Many tracking systems are available in the art, and the presentinvention is not to be limited to any particular tracking system. Inaddition, many systems are available for merging data obtained fromdifferent imaging modalities. However, an additional advantage of thesystem described herein is that the optical camera of the trackingsystem is typically in a fixed orientation related to the object to beimaged; i.e., placing the camera on the gantry as shown in the figuresprovides a reference plane for the tomosynthesis images that facilitatesmapping of the coordinate planes for ultrasound. It should be noted,however, that it is not a requirement of the invention that the opticalcamera of the tracking system be mounted on the gantry.

Tracking system 15 comprises optical camera 20 and a plurality ofoptical transmitters 250, shown in FIG. 4A mounted on an ultrasoundprobe 204, and in FIG. 4B mounted to a biopsy hand piece; however,skilled persons will understand that alternative tracking systems can beused, such as RF magnetic tracking systems. Optical camera 20 isconnected to communication network 12 for transmitting the threedimensional coordinate data of the plurality of optical transmitters tonavigation system 13 in the tracker co-ordinate space. Optical camera 20monitors the position and orientation of ultrasound probe 204 bytracking ultrasound transmitters 250 and transmits this data tonavigation system 13 via communication network 12. Skilled persons willappreciate that in some alternative embodiments, optical camera 20 canbe connected directly to navigation system 13 via a direct communicationlink, which may be a physical communication link or a wirelesscommunication link.

In the embodiment shown, ultrasound probe 204 is removably engaged toultrasound tracker 208 which has ultrasound transmitters 250 that aretracked by optical camera 20 in the tracker co-ordinate space. Skilledpersons will appreciate that while in the embodiment shown, ultrasoundtransmitters 250 are optical transmitters tracked by optical camera 20,other transmitter-receiver systems can be used. For example, in otherembodiments, RF transmitters and receivers can be used to track theposition and orientation of ultrasound probe 204 in the trackerco-ordinate space. Additionally, skilled persons will appreciate thatother orientations and positions of ultrasound transmitters 250 onultrasound tracker 208 can be used to provide position and orientationinformation detectable by optical camera 20 and transmitted tonavigation system 13. Skilled persons will understand that the use oftransmitters that are removably connected to ultrasound probe 204 cantend to provide the ability to configure any ultrasound probe with anyshape of transducer, such as linear transducers, curvilinear transducersand array and phased array transducers.

One aspect of the navigation system involves registering the field ofview of the ultrasound probe (or, in the case of the biopsy needle, thedistal tip of the needle) with the navigation system. The '633application describes the use of a stylus, which is used to providecertain information to the navigation system related to the width andcurvature of the ultrasound probe, and thus the field of view. Asdescribed in the '633 application, alternative methods of configuringthe probe, including using a pre-generated calibration matrix, may alsobe used. Similar configuration matrices can be used for interventionaldevices, such as biopsy needles, where the configuration matrices arecustomized according to the physical characteristics of the devices. Ingeneral, the configuration matrices provide information related todistances between each of the transmitters that are mounted to thedevice/probe and the relevant points on the device/probe to enable thenavigation system to orient and register received images.

In one embodiment, it is envisioned that the 3-dimensional image dataand the ultrasound image feed may be used to enable a guided biopsy ofthe breast, as it is immobilized by the positioning paddle. The biopsyneedle may be supported by hand, or via a lateral arm support, and mayaccess the breast from the side or, in configurations where the acousticgel can be kept away from the biopsy entry point, through the meshitself.

In an additional embodiment it is realized that the co-registration ofthe ultrasound image feed with the tomosynthesis slices facilitatescapture of ultrasound images at different planes that correspond to thetomosynthesis image planes. Captured ultrasound images, each acquiredfrom parallel planes within the breast, can be reconstructed to generatea three dimensional volume of ultrasound data. The present invention canbe used to navigate the ultrasound probe to an appropriate locationwithin a three dimensional volume acquired using tomosynthesis. Once theprobe is in the appropriate location, in relating to the threedimensional volume, a sequence of ultrasound images may be obtained atvarying depths by varying the strength of the ultrasound signal, therebygenerating the information for the corresponding ultrasound imagevolume. It should be recognized that the present invention is notlimited to generation of such a three dimensional ultrasound imagevolume at any particular location during an examination.

For example, FIG. 5 illustrates the different coordinate planes of theultrasound transducer (in a particular position), and the 3-Dtomosynthesis imaging geometry, indicated generally by the cone beamoriginating from the x-ray tube head. The tracking system usesinformation provided by the transmitters on the probe to determine theorientation and position of the ultrasound probe in real-time, andtherefore provides information related to the particular viewing planeassociated with ultrasound feed at any instant.

The real time information provided via the ultrasound image feed can beused in a variety of manners. In one embodiment, after a region ofinterest in the 3 Dimensional data set is identified, i.e., by selectinga region of interest on a slice, for example, the 3-D image may be heldstable and may be displayed at a workstation. Manual manipulation of theultrasound probe may be performed, with ultrasound image feed beingprovided on the ultrasound display (or a portion of the workstationdisplay allocated to the ultrasound image). As the ultrasound probeapproaches the region of interest, (i.e., as the ultrasound probe ismoved into a position which corresponds to the x, y, z coordinates orthe ROI in the 3D volume), a visual or audible cue may be provided tothe user, allowing the user to view the mass in the different modality,and/or capture a representative image for later review.

In another embodiment, calibration matrices and dynamic reconstructiontechniques can be used to dynamically reconstruct the tomosynthesisvolume along the imaging plane of the ultrasound transducer. It isappreciated that a tomosynthesis volume set is quite large, and it maynot be practical to perform a continuous reconstruction in this manner,but rather provide a user interface that enables the user to reconstructthe selected plane on-the-fly should the medical professional identify aregion of interest in the ultrasound image feed. A method forreconstructing tomosynthesis volumes along variable planes is describedin more detail in U.S. Patent Ser. No. 61/556,384, filed Nov. 7, 2011and entitled “SYSTEM AND METHOD FOR SELECTABLE PLANE RECONSTRUCTION OFBREAST TOMOSYNTHESIS IMAGES”, incorporated herein by reference.

FIG. 6 illustrates an ultrasound probe having transmitters mountedthereon, performing an ultrasound scan on a phantom while the phantom ispositioned upon a housing of a detector of a three dimensionaltomosynthesis system.

FIG. 7 is a flow diagram illustrating one exemplary sequence of steps ina process 700 for examining a patient using multiple different imagingmodalities for diagnosis purposes while the patient remains in a fixedposition. At step 70 the patient is immobilized, for example using thepositioning paddle as described above. At step 72 the 3-D image data iscaptured, for example using tomosynthesis, CT, PET, SPECT, gamma orother imaging technique. This 3-D image data may be displayed on adisplay device, which is viewable by a user, enabling a user to selectan area of interest for further review. At step 74, the region ofinterest (ROI) is identified or otherwise selected. Selection may bemanual, or may automatically be selected using computer assisteddetection (CAD) software, or may include a combination of bothtechniques. At step 76, while remaining immobilized, is prepared forultrasound imaging. For example, acoustic couplant may be applied to thesurface of the positioning paddle. Step 78, the ‘diagnosis’ step,involves scanning the breast using the ultrasound probe to acquireimages and comparing the images against the selected ROI until the ROIis viewed in both modalities. Once diagnosis is completed, at step 80the positioning paddle may be withdrawn and the patient may be releasedfrom their position.

FIG. 8 illustrates is a pictorial representation of equipment which maybe included in a radiology suite capable of performing breast cancerscreening and diagnosis while a patient remains in one upright, fixedposition. The suite includes a tomosynthesis acquisition system 10, anultrasound system 14 and a workstation 800. The navigation system andtracking system may be embodied in hardware devices such as thosedescribed above with regard to FIGS. 1-7 , and software which may beembodied as computer programs loaded onto the workstation 800, which areoperable when executed on by a process of the workstation to perform thetasks identified above.

Accordingly, a multi-modality cancer screening and diagnosis system hasbeen shown and described that allows a cancer screening and diagnosis ofa patient using at least two different and sequential three-dimensionalimaging techniques without the need to reposition the patient. Apositioning paddle suitable for use with two different and distinctimaging modalities allows a patient to remain in a fixed positionthroughout a multi-mode imaging diagnostic process. Tracking andnavigation software facilitate image guided diagnosis. As a result, thespeed and accuracy of diagnosis may be greatly improved.

Having described several exemplary embodiments, it will be appreciatedthat numerous specific details have been set forth in order to provide athorough understanding of the embodiments described herein. However, itwill be understood by those of ordinary skill in the art that theembodiments described herein may be practiced without these specificdetails. In other instances, well-known methods, procedures andcomponents have not been described in detail so as not to obscure theembodiments described herein. Furthermore, this description is not to beconsidered as limiting the scope of the embodiments described herein inany way, but rather as merely describing the implementation of thevarious embodiments described herein.

The embodiments of the systems and methods that have been describedherein may be implemented in hardware or software, or a combination ofboth. In an embodiment these systems and methods are implemented incomputer programs executing on programmable computers each comprising atleast one processor, a data storage system (including volatile andnon-volatile memory and/or storage elements), at least one input device,and at least one output device. For example and without limitation, theprogrammable computers may be a mainframe computer, server, personalcomputer, laptop, personal data assistant, or cellular telephone.Program code is applied to input data to perform the functions describedherein and generate output information. The output information isapplied to one or more output devices, in known fashion.

Each program can be implemented in a high level procedural or objectoriented programming and/or scripting language to communicate with acomputer system. However, the programs can be implemented in assembly ormachine language, if desired. In any case, the language may be acompiled or interpreted language. Each such computer program can bestored on a storage media or a device (e.g. ROM or magnetic diskette)readable by a general or special purpose programmable computer, forconfiguring and operating the computer when the storage media or deviceis read by the computer to perform the procedures described herein. Theembodiments may also be considered to be implemented as acomputer-readable storage medium, configured with a computer program,where the storage medium so configured causes a computer to operate in aspecific and predefined manner to perform the functions describedherein.

Furthermore, the system, processes and methods of the describedembodiments are capable of being distributed in a computer programproduct comprising a computer readable medium that bears computer usableinstructions for one or more processors. The medium may be provided invarious forms, including one or more diskettes, compact disks, tapes,chips, wireline transmissions, satellite transmissions, internettransmission or downloadings, magnetic and electronic storage media,digital and analog signals, and the like. The computer useableinstructions may also be in various forms, including compiled andnon-compiled code.

Therefore, having described numerous embodiments, it is understood thatthe present invention is not limited merely to those embodiment, butincludes rather includes equivalents thereto. The invention shouldtherefore be only be limited by the attached claims.

What we claim is:
 1. A method of examining a breast of a patient, themethod comprising: immobilizing the breast; and while the breast isimmobilized: acquiring at least one first image of the breast using afirst imaging modality having a first coordinate space; selecting aregion of interest (ROI) in the at least one first image; identifyingcoordinates of the ROI in the at least one first image and with respectto the first coordinate space; tracking a position and an orientation ofa second imaging modality to determine a field of view of the secondimaging modality having a second coordinate space, the second coordinatespace being rotationally different than the first coordinate space;acquiring at least one second image of the breast using the secondimaging modality and with respect to the second coordinate space;locating the ROI in the at least one second image; displaying the atleast one second image based on the field of view of the second imagingmodality and reformatting the at least one first image to a sameposition and a same orientation as the field of view of the secondimaging modality; and releasing the breast from immobilization.
 2. Themethod of claim 1, wherein the first imaging modality is selected from agroup of imaging modalities including tomosynthesis, computedtomography, SPECT, PET and gamma imaging.
 3. The method of claim 1,wherein the second imaging modality includes ultrasound imaging.
 4. Themethod of claim 1, wherein locating the ROI is performed manually. 5.The method of claim 1, wherein locating the ROI is performed withComputer Assisted Detection (CAD).
 6. The method of claim 1, whereinimmobilizing the breast comprises positioning a positioning paddle overthe breast, the positioning paddle comprising a porous fabric.
 7. Themethod of claim 6 wherein immobilizing the breast comprises tighteningthe fabric over the breast.
 8. The method of claim 1, wherein acquiringthe at least one second image using the second imaging modalitycomprises determining a location of an imaging probe in a coordinateplane of the at least one first image of the first imaging modality. 9.The method of claim 8, further comprising providing a visual or audiblecue when the imaging probe is directed to the coordinates of the ROI.10. The method of claim 8, further comprising determining an orientationplane of the imaging probe, and reconstructing a three dimensionalvolume along the orientation plane of the imaging probe.
 11. The methodof claim 1, wherein at least one of the first imaging modality and thesecond imaging modality comprise a three dimensional imaging modality.12. The method of claim 1, wherein both of the first imaging modalityand the second imaging modality comprise a three dimensional imagingmodality.
 13. The method of claim 1, wherein tracking the position andthe orientation of the second imaging modality comprises tracking aposition of transmitters mounted on an ultrasound imaging probe.